1. Field of the Invention
The present invention relates to an image processing method that performs an image restoration processing of an image.
2. Description of the Related Art
An image (a taken image) that is obtained by an image pickup apparatus contains a blur component of an image caused by each aberration such as spherical aberration, coma aberration, distortion, or astigmatism of an image pickup optical system, and therefore it is deteriorated. The blur component of the image caused by such aberrations means that a light beam emitted from one point of an object is actually spread although it should be ideally collected on one point on an imaging plane again when any aberration does not exist and also any influence by diffractions does not exist, which is represented by point spread function (PSF).
The optical transfer function (OTF) that is obtained by performing the Fourier transform is frequency component information of the aberration, and it is represented by a complex number. An absolute value of the optical transfer function OTF, i.e. amplitude component is referred to as an MTF (Modulation Transfer Function), and a phase component is referred to as a PTF (Phase Transfer Function). The amplitude component MTF and the phase component PTF are frequency characteristics of an amplitude component and a phase component of the image deterioration caused by the aberration, respectively, and the phase component is represented as a phase angle by the following expression.PTF=tan−1(Im(OTF)/Re(OTF))
In the expression, Re(OTF) and Im(OTF) indicate a real part and an imaginary part of the optical transfer function OTF, respectively. Thus, the optical transfer function OTF of the image pickup optical system deteriorates the amplitude component MTF and the phase component PTF of the image, and therefore the deteriorated image is in a state where each point of the object is asymmetrically blurred as it has the coma aberration. Chromatic aberration of magnification is generated because an imaging position is shifted by the difference of an imaging magnification for each light wavelength and it is obtained as color components of for example RGB in accordance with the spectroscopic characteristics of an image pickup apparatus. Accordingly, the imaging positions are shifted between the RGB, and therefore the shift of the imaging position for a wavelength, i.e. the spread of the image caused by the phase shift is also generated in each color component.
As a method of correcting the deteriorations of the amplitude component MTF and the phase component PTF, a method of performing the correction using information of the optical transfer function OTF of the image pickup optical system is known. This method is called an image restoration or an image recovery, and hereinafter, a processing of correcting the deterioration of the image using the information of the optical transfer function (OTF) of the image pickup optical system is referred to as an image restoration processing. As described below in detail, as one of methods of the image restoration, a method of convoluting an image restoration filter that has inverse characteristics of the optical transfer function (OTF) with an input image is known.
In order to effectively use the image restoration, it is necessary to obtain more precise OTF information of the image pickup optical system. A general OTF of the image pickup optical system significantly varies in accordance with an image height (a position of an image). Furthermore, the optical transfer function OTF is two-dimensional data, which contains a real part and an imaginary part since it is a complex number. In addition, when an image restoration processing is performed for a color image that has three color components of RGB, the OTF data for one image height is tap number in a vertical direction×tap number in a horizontal direction×2 (the real part and the imaginary part)×3 (RGB). The tap number means a size of the OTF data in the vertical and horizontal directions. If these are stored with respect to all image pickup conditions such as an image height, an F-number (an aperture), zoom (a focal length), an object distance, an amount of data is enormous.
Japanese Patent Laid-Open No. 2010-56992 discloses a technology that stores a filter coefficient for correcting the deterioration of the image to perform an image processing. However, since a restoration filter depending on a position in a screen is necessary, an amount of data is enormous. Japanese Patent Laid-Open No. 2004-241991 discloses a method of obtaining a distance from an optical center for correcting a chromatic aberration of magnification and substituting the distance in a cubic function to calculate a correction moving amount of R and B components to determine a correction amount depending on the position in the screen though it is not a technology of the image restoration.
However, since the image restoration filter is two-dimensional data, the image restoration filter depending on the position of the image is not determined only by the distance from the optical center (a center of the image or an optical axis of the image pickup optical system), and a rotation is necessary. Therefore, the method of Japanese Patent Laid-Open No. 2004-241991 cannot be applied to the image restoration processing. Furthermore, since the coefficient of the image restoration filter varies minutely between taps, a value of the filter coefficient is significantly collapsed and the effect of the image restoration processing is not obtained.